HDR displays having light estimating controllers

ABSTRACT

A display has a screen which incorporates a light modulator. The screen may be a front projection screen or a rear-projection screen. The screen is illuminated with light from a light source comprising an array of controllable light-emitters. The controllable-emitters and elements of the light modulator may be controlled to adjust the intensity of light emanating from corresponding areas on the screen. The display may provide a high dynamic range.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.10/507,460 (accorded filing date 10 Sep. 2004), now U.S. Pat. No.7,403,332, which is the U.S. National Stage of International ApplicationNo. PCT/CA03/00350 filed 13 Mar. 2003, which claims the benefit of thefiling date of U.S. provisional patent application No. 60/363,563 filed13 Mar. 2002 and entitled HIGH DYNAMIC RANGE DISPLAY DEVICES.

The claimed invention was made as a result of activities undertakenwithin the scope of a joint research agreement as defined under 35U.S.C. 103(c) between the National Sciences and Engineering ResearchCouncil of Canada, Lorne Whitehead of the University of BritishColumbia, Wolfgang Stuerzlinger and Hugh Wilson of York University, andAvi Chaudhuri of McGill University.

TECHNICAL FIELD

The invention relates to displays for displaying digital images.

BACKGROUND

Dynamic range is the ratio of intensity of the highest luminance partsof a scene and the lowest luminance parts of a scene. For example, theimage projected by a video projection system may have a maximum dynamicrange of 300:1.

The human visual system is capable of recognizing features in sceneswhich have very high dynamic ranges. For example, a person can look intothe shadows of an unlit garage on a brightly sunlit day and see detailsof objects in the shadows even though the luminance in adjacent sunlitareas may be thousands of times greater than the luminance in the shadowparts of the scene. To create a realistic rendering of such a scene canrequire a display having a dynamic range in excess of 1000:1. The term“high dynamic range” means dynamic ranges of 800:1 or more.

Modern digital imaging systems are capable of capturing and recordingdigital representations of scenes in which the dynamic range of thescene is preserved. Computer imaging systems are capable of synthesizingimages having high dynamic ranges. However, current display technologyis not capable of rendering images in a manner which faithfullyreproduces high dynamic ranges.

Blackham et al., U.S. Pat. No. 5,978,142 discloses a system forprojecting an image onto a screen. The system has first and second lightmodulators which both modulate light from a light source. Each of thelight modulators modulates light from the source at the pixel level.Light modulated by both of the light modulators is projected onto thescreen.

Gibbon et al., PCT application No. PCT/US01/21367 discloses a projectionsystem which includes a pre modulator. The pre modulator controls theamount of light incident on a deformable mirror display device. Aseparate pre-modulator may be used to darken a selected area (e.g. aquadrant).

There exists a need for cost effective displays capable of reproducing awide range of light intensities in displayed images.

SUMMARY OF THE INVENTION

This invention provides displays for displaying images. One embodimentof the invention provides a display comprising: a light sourcecomprising an array of light-emitting elements. Each of the elements hasa controllable light output; and, a spatial light modulator comprising aplurality of controllable elements located to modulate light from thelight source. A diffuser directs light from the light source which hasbeen modulated by the spatial light modulator to a viewing area.

Another aspect of the invention provides a display comprising:

a spatial light modulator comprising an array of controllable elements,each of the controllable elements providing a controllable lighttransmission; a light source comprising an array of solid statelight-emitting elements each located to illuminate a plurality ofcorresponding controllable elements of the spatial light modulator andeach having a controllable light output; and, a diffuser. Brightness ofa point on the diffuser may be controlled by controlling the lightoutput of one of the light-emitting elements corresponding to the pointand controlling the light transmission of one of the controllableelements corresponding to the point.

A further aspect of the invention provides a display comprising: lightprovision means for providing light spatially modulated at a firstspatial resolution; spatial modulation means for further spatiallymodulating the light at a second resolution different from the firstresolution; and, means for controlling the first and second spatialmodulation means to display an image defined by image data.

The invention also provides a method for displaying an image. The methodcomprises controlling an array of individually-controllablelight-emitting elements to have brightnesses determined by a first setof image data; illuminating a face of a spatial light modulator withlight from the array of light-emitting elements, the spatial lightmodulator comprising an array of elements, each of the elements having acontrollable transmissivity; and, controlling the transmissivity of theelements of the spatial light modulator with a second set of image data.

Further aspects of the invention and features of specific embodiments ofthe invention are described below.

BRIEF DESCRIPTION OF THE DRAWINGS

In drawings which illustrate non-limiting embodiments of the invention,

FIG. 1 is a schematic illustration of a display according to oneembodiment of the invention;

FIG. 1A is a schematic illustration of a specific implementation of thedisplay of FIG. 1;

FIG. 2 is a schematic illustration of a display according to analternative embodiment of the invention comprising four spatial lightmodulators;

FIG. 3 is a schematic illustration of a rear-projection-type displayaccording to a further embodiment of the invention;

FIG. 4 is a schematic illustration of a front-projection-type displayaccording to a still further embodiment of the invention;

FIG. 5 is a drawing illustrating a possible relationship between pixelsin a higher-resolution spatial light modulator and pixels in alower-resolution spatial light modulator in a display according to theinvention;

FIG. 5A illustrates an effect of providing one light modulator which haslower resolution than another light modulator;

FIG. 6 is a schematic illustration of a front-projection-type colordisplay having an alternative projector construction;

FIGS. 6A and 6B are expanded cross-sectional views of portions of thefront-projection screen of the color display of FIG. 6;

FIG. 7 is a graph illustrating how light imaged onto a higher-resolutionlight modulator from pixels of a lower-resolution light modulator canoverlap to yield a smooth variation in light intensity with position;

FIG. 7A is a graph illustrating how the variation in light intensitywith position for the image of a pixel of a light modulator can berepresented as the convolution of a square profile and a spreadfunction;

FIG. 8 is a schematic cross-section of a display according to analternative embodiment of the invention and FIG. 8A is a schematic frontview thereof;

FIG. 8B is a schematic cross section of a display in which a spatiallight modulator is spaced in front of an array of light sources;

FIG. 8C is a schematic view of a display having a grid interposedbetween an array of light sources and a spatial light modulator;

FIG. 8D is an isometric view of a hexagonal grid;

FIG. 8E is a schematic representation of one channel through a gridillustrating reflected and non-reflected light components impinging on aspatial light modulator;

FIG. 8F is a graph showing how reflected and non-reflected lightcomponents can sum to provide improved uniformity of illumination;

FIG. 8G is a schematic representation of a display wherein internallyreflecting members which form a grid are formed integrally with thematerial encapsulating LEDs;

FIGS. 9A and 9B illustrate two possible configurations for an array oflight emitting elements which could be used in the embodiment of FIG. 8;

FIG. 9C illustrates the use of light barriers to provide increasedsharpness;

FIG. 10 is a schematic illustration of a projection-type displayaccording to an alternative embodiment of the invention;

FIG. 11 is a block diagram of a calibration mechanism;

FIG. 11A is a depiction of an LED illustrating paths by which straylight exits the LED; and,

FIGS. 11B, 11C, 11D and 11E are schematic diagrams of alternativecalibration mechanisms.

DESCRIPTION

Throughout the following description, specific details are set forth inorder to provide a more thorough understanding of the invention.However, the invention may be practiced without these particulars. Inother instances, well known elements have not been shown or described indetail to avoid unnecessarily obscuring the invention. Accordingly, thespecification and drawings are to be regarded in an illustrative, ratherthan a restrictive, sense.

This invention provides displays capable of rendering images with highdynamic ranges. Displays according to the invention comprise two lightmodulating stages. Light passes through the stages in series to providean image which has an increased dynamic range.

FIG. 1 illustrates schematically a display 10 according to a simpleembodiment of the invention. The sizes of elements and distances betweenthem in FIG. 1 are not to scale. Display 10 comprises a light source 12.Light source 12 may, for example, comprise a projection lamp such as anincandescent lamp or an arc lamp, a laser, or another suitable source oflight. Light source 12 may comprise an optical system comprising one ormore mirrors, lenses or other optical elements which cooperate todeliver light to the rest of display 10.

In the illustrated embodiment, light from light source 12 is directedtoward a first light modulator 16. Light source 12 preferably providessubstantially uniform illumination of first light modulator 16. Lightmodulator 16 comprises an array of individually addressable elements.Light modulator 16 may comprise, for example, a LCD (liquid crystaldisplay), which is an example of a transmission-type light modulator ora DMD (deformable mirror device), which is an example of areflection-type light modulator. Display driver circuitry (not shown inFIG. 1) controls the elements of light modulator 16 according to datawhich defines an image being displayed.

Light which has been modulated by first light modulator 16 is imagedonto a rear-projection screen 23 by a suitable optical system 17. Lightfrom a small area of first light modulator 16 is directed by opticalsystem 17 to a corresponding area on rear-projection screen 23. In theillustrated embodiment, optical system 17 comprises a lens having afocal length ƒ. In general, the optical system 17 which images lightmodulated by first light modulator 16 onto rear-projection screen 23 maycomprise one or more mirrors, lenses or other optical elements. Such anoptical system has the function of imaging light modulated by the firstlight modulator onto a second light modulator. Optical system 17 may betermed an imaging means.

In the illustrated embodiment, rear-projection screen 23 comprises asecond light modulator 20 and a collimator 18. A main function ofcollimator 18 is to cause light which passes through rear-projectionscreen 23 to be directed preferentially toward a viewing area.Collimator 18 may comprise a Fresnel lens, a holographic lens, or, inthe alternative, another arrangement of one or more lenses and/or otheroptical elements which operate to guide light in the direction of aviewing area.

In the illustrated embodiment, collimator 18 causes light to travelthrough the elements of second light modulator 20 in a direction whichis generally normal to screen 23. As light incident from collimator 18travels through second light modulator 20 it is further modulated. Thelight then passes to a diffuser 22 which scatters the outgoing lightthrough a range of directions so that a viewer located on an oppositeside of diffuser 22 from first light modulator 16 can see lightoriginating from the whole area of screen 23. In general, diffuser 22may scatter light to a different angular extent in the horizontal andvertical planes. Diffuser 22 should be selected so that light modulatedby second light modulator 20 is scattered through a range of angles suchthat the maximum scatter angle is at least equal to the angle subtendedby screen 23 when viewed from a desired viewing location.

Rear-projection screen 23 may differ in area from first light modulator16. For example, rear-projection screen 23 may be larger in area thanfirst light modulator 16. Where this is the case, optical system 17expands the beam of light modulated by first light modulator 16 toilluminate a corresponding area on rear-projection screen 23 which islarger than first light modulator 16.

Second light modulator 20 may be of the same type as first lightmodulator 16 or a different type. Where first and second lightmodulators 16 and 20 are both of types that polarize light, second lightmodulator 20 should, as much as is practical, be oriented so that itsplane of polarization matches that of the light incident on it fromfirst light modulator 16.

Display 10 may be a color display. This may be achieved in various waysincluding:

-   -   making one of first light modulator 16 and second light        modulator 20 a color light modulator;    -   providing a plurality of different first light modulators 16        operating in parallel on different colors; and,    -   providing a mechanism for rapidly introducing different color        filters into the light path ahead of second light modulator 20.        As an example of the first approach above, second light        modulator 20 may comprise an LCD panel having a plurality of        pixels each comprising a number of colored sub-pixels. For        example, each pixel may comprise three sub-pixels, one        associated with a red filter, one associated with a green filter        and one associated with a blue filter. The filters may be        integral with the LCD panel.

As shown in FIG. 1A, Light source 12, first light modulator 16 andoptical system 17 may all be parts of a digital video projector 37located to project an image defined by a signal 38A from a controller 39onto the back side of rear-projection screen 23. The elements of secondlight modulator 20 are controlled by a signal 38B from controller 39 toprovide an image to a viewer which has a high dynamic range.

Controller 39 may comprise any suitable data processor. Controller 39may comprise one or more microprocessors running suitable controlsoftware together with interfaces which permit controller 39 to controlthe operation of a display according to the invention. The generalconstruction of such controllers and general techniques for programmingsuch controllers to provide desired functions are well known to thoseskilled in the art and will not be described in detail herein.

As shown in FIG. 2, a display 10A according to the invention maycomprise one or more additional light modulation stages 24. Eachadditional light modulation stage 24 comprises a collimator 25, a lightmodulator 26 and an optical system 27 which focuses light from lightmodulator 26 onto either the next additional light modulation stage 24or on collimator 18. In device 10A of FIG. 2 there are two additionallight modulation stages 24. Devices according to this embodiment of theinvention may have one or more additional light modulation stages 24.

The luminance of any point on output diffuser 22 can be adjusted bycontrolling the amount of light passed on by corresponding elements oflight modulators 16, 20 and 26. This control may be provided by asuitable control system (not shown in FIG. 2) connected to drive each oflight modulators 16, 20 and 26.

As noted above, light modulators 16, 20 and 26 may all be of the sametype or may be of two or more different types. FIG. 3 illustrates adisplay 10B according to an alternative embodiment of the inventionwhich includes a first light modulator 16A which comprises a deformablemirror device. A deformable mirror device is a “binary” device in thesense that each pixel may be either “on” or “off”. Different apparentbrightness levels may be produced by turning a pixel on and off rapidly.Such devices are described, for example, in U.S. Pat. Nos. 4,441,791and, 4,954,789 and are commonly used in digital video projectors. Lightsource 12 and first light modulator 16 (or 16A) may be the light sourceand modulator from a commercial digital video projector, for example.

FIG. 4 illustrates a front-projection-type display 10C according to theinvention. Display 10C comprises a screen 34. A projector 37 projects animage 38 onto screen 34. Projector 37 comprises a suitable light source12, a first light modulator 16 and an optical system 17 suitable forprojecting an image defined by first light modulator 16 onto screen 34.Projector 37 may comprise a commercially available display projector.Screen 34 incorporates a second light modulator 36. Second lightmodulator 36 comprises a number of addressable elements which can beindividually controlled to affect the luminance of a corresponding areaof screen 34.

Light modulator 36 may have any of various constructions. For example,light modulator 36 may comprise an array of LCD elements each having acontrollable transmissivity located in front of a reflective backing.Light projected by projector 37 passes through each LCD element and isreflected back through the LCD element by the reflective backing. Theluminance at any point on screen 34 is determined by the intensity oflight received at that point by projector 37 and the degree to whichlight modulator 36 (e.g. the LCD element at that point) absorbs lightbeing transmitted through it.

Light modulator 36 could also comprise an array of elements havingvariable retro-reflection properties. The elements may be prismatic.Such elements are described, for example, in Whitehead, U.S. Pat. No.5,959,777 entitled Passive High Efficiency Variable Reflectivity ImageDisplay Device and, Whitehead et al., U.S. Pat. No. 6,215,920 entitledElectrophoretic, High Index and Phase Transition Control of TotalInternal Reflection in High Efficiency Variable Reflectivity ImageDisplays.

Light modulator 36 could also comprise an array of electrophoreticdisplay elements as described, for example, in Albert et al., U.S. Pat.No. 6,172,798 entitled Shutter Mode Microencapsulated ElectrophoreticDisplay; Comiskey et al., U.S. Pat. No. 6,120,839 entitledElectro-osmotic Displays and Materials for Making the Same; Jacobson,U.S. Pat. No. 6,120,588 entitled: Electronically AddressableMicroencapsulated Ink and Display; Jacobson et al., U.S. Pat. No.6,323,989 entitled Electrophoretic Displays Using Nanoparticles; Albert,U.S. Pat. No. 6,300,932 entitled Electrophoretic Displays withLuminescent Particles and Materials for Making the Same or, Comiskey etal., U.S. Pat. No. 6,327,072 entitled Microcell ElectrophoreticDisplays.

As shown in FIGS. 6A and 6B, screen 34 preferably comprises a lenselement 40 which functions to direct light preferentially toward theeyes of viewers. In the illustrated embodiment, lens element 40comprises a Fresnel lens having a focal point substantially coincidentwith the apex of the cone of light originating from projector 37. Lenselement 40 could comprise another kind of lens such as a holographiclens. Lens element 40 incorporates scattering centers 45 which provide adesired degree of diffusion in the light reflected from screen 34. Inthe illustrated embodiment, second light modulator 36 comprises areflective LCD panel having a large number of pixels 42 backed by areflective layer 43 and mounted on a backing 47.

Where light modulator 36 comprises an array of elements having variableretro-reflection properties, the elements themselves could be designedto direct retro-reflected light preferentially in a direction of aviewing area in front of screen 34. Reflective layer 43 may be patternedto scatter light to either augment the effect of scattering centers 45or replace scattering centers 45.

As shown in FIG. 4, a controller 39 provides data defining image 38 toeach of first light modulator 16 and second light modulator 36.Controller 39 could comprise, for example, a computer equipped with asuitable display adapter. The luminance of any point on screen 34 isdetermined by the combined effect of the pixels in first light modulator16 and second light modulator 36 which correspond to that point. Thereis minimum luminance at points for which corresponding pixels of thefirst and second light modulators are set to their “darkest” states.There is maximum luminance at points for which corresponding pixels ofthe first and second light modulators are set to their “brightest”states. Other points have intermediate luminance values. The maximumluminance value might be, for example, on the order of 10⁵ cd/m². Theminimum luminance value might be, for example on the order of 10⁻²cd/m².

The cost of a light modulator and its associated control circuitry tendsto increase with the number of addressable elements in the lightmodulator. In some embodiments of the invention one of the lightmodulators has a spatial resolution significantly higher than that ofone or more other ones of the light modulators. When one or more of thelight modulators are lower-resolution devices the cost of a displayaccording to such embodiments of the invention may be reduced. In colordisplays comprising two or more light modulators, one of which is acolor light modulator (a combination of a plurality of monochrome lightmodulators may constitute a color light modulator as shown, for example,in FIG. 6) and one of which is a higher-resolution light modulator, thehigher-resolution light modulator should also be the color lightmodulator. In some embodiments the higher-resolution light modulator isimaged onto the lower-resolution light modulator. In other embodimentsthe lower-resolution light modulator is imaged onto thehigher-resolution light modulator.

FIG. 5 illustrates one possible configuration of pixels in a display 10as shown in FIG. 1. Nine pixels 42 of a second light modulator 20correspond to each pixel 44 of a first light modulator 16. The number ofpixels 42 of second light modulator 20 which correspond to each pixel 44of first light modulator 16 may be varied as a matter of design choice.Pixels 44 of the higher-resolution one of first and second lightmodulators 16 and 20 (or 36) should be small enough to provide a desiredoverall resolution. In general there is a trade off between increasingresolution and increasing cost. In a typical display thehigher-resolution light modulator will provide an array of pixels havingat least a few hundred pixels in each direction and more typically over1000 pixels in each direction.

The size of pixels 42 of the lower-resolution one of the first andsecond light modulators determines the scale over which one can reliablygo from maximum intensity to minimum intensity. Consider, for example,FIG. 5A which depicts a situation where one wishes to display an imageof a small maximum-luminance spot on a large minimum-luminancebackground. To obtain maximum luminance in a spot 47, those pixels ofeach of the first and second light modulators which correspond to spot47 should be set to their maximum-luminance values. Where the pixels ofone light modulator are lower in resolution than pixels of the otherlight modulator then some pixels of the lower-resolution light modulatorwill straddle the boundary of spot 47. This is the case, for example, inFIG. 5A.

Outside of spot 47 there are two regions. In region 48 it is notpossible to set the luminance to its minimum value because in thatregion the lower-resolution light modulator is set to its highestluminance value. In region 49 both of the light modulators can be set totheir lowest-luminance values. If, for example, each of the first andsecond light modulators has a luminance range of 1 to 100 units, thenregion 47 might have a luminance of 100×100=10,000 units, region 48would have a luminance of 100×1=100 units and region 49 would have aluminance of 1×1=1 units.

As a result of having one of the light modulators lower in resolutionthan the other, each pixel of the lower-resolution light modulatorcorresponds to more than one pixel in the higher-resolution lightmodulator. It is not possible for points corresponding to any one pixelof the lower-resolution light modulator and different pixels of thehigher-resolution light modulator to have luminance values at extremesof the device's dynamic range. The maximum difference in luminancebetween such points is determined by the dynamic range provided by thehigher-resolution light modulator.

It is generally not a problem that a display is not capable of causingclosely-spaced points to differ in luminance from one another by thefull dynamic range of the display. The human eye has enough intrinsicscatter that it is incapable of appreciating large changes in luminancewhich occur over very short distances in any event.

In a display according to the invention which includes both alower-resolution spatial light modulator and a higher-resolution spatiallight modulator, controller 39 may determine a value for each pixel ofthe lower-resolution spatial light modulator and adjust the signalswhich control the higher-resolution spatial light modulator to reduceartefacts which result from the fact that each pixel of thelower-resolution spatial light modulator is common to a plurality ofpixels of the higher-resolution spatial light modulator. This may bedone in any of a wide number of ways.

To take but one example, consider the case where each pixel of thelower-resolution spatial light modulator corresponds to a plurality ofpixels of the higher-resolution spatial light modulator. Image dataspecifying a desired image is supplied to the controller. The image dataindicates a desired luminance for an image area corresponding to each ofthe pixels of the higher-resolution spatial light modulator. Thecontroller may set the pixels of the lower-resolution light modulator toprovide an approximation of the desired image. This could beaccomplished, for example, by determining an average or weighted averageof the desired luminance values for the image areas corresponding toeach pixel of the lower-resolution display.

The controller may then set the pixels of the higher-resolution displayto cause the resulting image to approach the desired image. This couldbe done, for example, by dividing the desired luminance values by theintensity of light incident from the lower-resolution light modulator onthe corresponding pixels of the higher-resolution light modulator. Theintensity of light incident from the lower-resolution light modulator ona pixel of the higher-resolution light modulator can be computed fromthe known way that light from each pixel of the lower resolution spatiallight modulator is distributed on the higher resolution spatial lightmodulator. The contributions from one or more of the pixels of the lowerresolution spatial light modulator can be summed to determine theintensity with which any pixel of the higher resolution spatial lightmodulator will be illuminated for the way in which the pixels of thelower resolution spatial light modulator are set.

If the low-resolution pixels are too large then a viewer may be able todiscern a halo around bright elements in an image. The low resolutionpixels are preferably small enough that the appearance of bright patcheson dark backgrounds or of dark spots on bright backgrounds is notunacceptably degraded. It is currently considered practical to providein the range of about 8 to about 144, more preferably about 9 to 36,pixels on the higher-resolution light modulator for each pixel of thelower-resolution light modulator.

The sizes of steps in which each of pixels 42 and 44 can adjust theluminance of point(s) on the image are not necessarily equal. The pixelsof the lower-resolution light modulator may adjust light intensity incoarser steps than the pixels of the higher-resolution light modulator.For example, the lower-resolution light modulator may permit adjustmentof light intensity for each pixel over an intensity range of 1 to 512units in 8 steps while the higher-resolution light modulator may permitadjustment of the light intensity for each pixel over a similar range in512 steps. While pixels 42 and 44 are both illustrated as being squarein FIG. 5, this is not necessary. Pixels 42 and/or 44 could be othershapes, such as rectangular, triangular, hexagonal, round, or oval.

The pixels of the lower-resolution light modulator preferably emit lightwhich is somewhat diffuse so that the light intensity varies reasonablysmoothly as one traverses pixels of the lower-resolution lightmodulator. This is the case where the light from each of the pixels ofthe lower-resolution light modulator spreads into adjacent pixels, asshown in FIG. 7. As shown in FIG. 7A, the intensity profile of a pixelin the lower-resolution light modulator can often be approximated bygaussian spread function convolved with a rectangular profile having awidth d₁ equal to the active width of the pixel. The spread functionpreferably has a full width at half maximum in the range of 0.3×d₂ to3×d₂, where d₂ is the center-to-center inter-pixel spacing, to yield thedesired smoothly varying light intensity. Typically d₁ , is nearly equalto d₂.

In the embodiment of FIG. 5, each pixel 42 comprises three sub pixels43R, 43G and 43B (for clarity FIG. 5 omits sub pixels for some pixels42). Sub-pixels 43R, 43G and 43B are independently addressable. They arerespectively associated with red, green and blue color filters which areintegrated into second light modulator 20. Various constructions of LCDpanels which include a number of colored sub-pixels and are suitable foruse in this invention are known in the art.

For front projection-type displays (for example the display 10C of FIG.4), it is typically most practical for first light modulator 16 tocomprise a high-resolution light modulator which provides colorinformation and for light modulator 36 to comprise a monochrome lightmodulator. Light modulator 36 preferably has reasonably smalladdressable elements so that the boundaries of its elements do not forma visually distracting pattern. For example, light modulator 36 may havethe same number of addressable elements as projector 37 (although eachsuch element will typically have significantly larger dimensions thanthe corresponding element in light modulator 16 of projector 37).

Projector 37 may have any suitable construction. All that is required isthat projector 37 be able to project light which has been spatiallymodulated to provide an image onto screen 34. FIG. 6 illustrates adisplay system 10D according to a further alternative embodiment of theinvention. System 10D comprises a screen 34 which has an integratedlight modulator 36 as described above with reference to FIG. 4. System10D comprises a projector 37A which has separate light modulators 16R,16G and 16R for each of three colors. Light modulated by each of lightmodulators 16R, 16G and 16R is filtered by a corresponding one of threecolored filters 47R, 47G and 47B. The modulated light is projected ontoscreen 34 by optical systems 17. A single light source 12 may supplylight to all three light modulators 16R, 16G, and 16B, or separate lightsources (not shown) may be provided.

In the embodiments described above, light from a light source isspatially modulated by a first light modulator and then imaged onto asecond light modulator. The inventors have realized that the functionsof the light source and first light modulator can be combined byproviding a light source comprising an array of light-emitting elementswhich each have a controllable brightness. The light-emitting elementsmay be solid state devices. For example, the light-emitting elements maycomprise light-emitting diodes (LEDs). Each of the LEDs may be driven bya driver circuit which allows the current flowing through the LED, andconsequently the brightness of the light emitted by the LED, to becontrolled. The controller may also, or in the alternative, control aduty cycle of the corresponding LED. As discussed below, the drivingcircuit may monitor current being delivered to each LED or each group ofLEDs and may generate an error signal if the magnitude of the currentbeing delivered to each LED or each group of LEDs has an unexpectedvalue. Such error signals may be used by a controller to compensate forfailed LEDs.

In a preferred embodiment of the invention, the LEDs are of a type whichemit white light. For example, the LEDs may comprise an array oftri-color LEDs. Tri-color LEDs which each include red, green and blueLEDs all encapsulated within a single housing are commerciallyavailable. One or more white LEDs may be used to illuminate each groupof pixels of the second light modulator.

FIG. 8 shows a section through a display 60 according to an embodimentof the invention in which a rear-projection screen 53 comprising adiffusing layer 22 is illuminated by an array 50 of LEDs 52. Thebrightness of each LED 52 is controlled by a controller 39. Screen 53includes a light modulator 20. The rear face of light modulator 20 isilluminated by LED array 50. FIG. 8A is a schematic front view of aportion of display 60 for a case where controllable elements (pixels) 42of light modulator 20 correspond to each LED 52. Each of thecontrollable elements 42 may comprise a plurality of colored sub-pixels.

LEDs 52 may be arranged in any suitable manner in array 50. Two likelyarrangements of LEDs 52 are shown in FIGS. 9A and 9B. FIG. 9Aillustrates a rectangular array 50A of light sources 51. FIG. 9Billustrates a hexagonal array 50B of light sources 51. Light sources 51may comprise LEDs 52. Where light sources 51 comprise discrete devices,a regular spacing between light sources 51 may be maintained by packinglight sources 51 together as illustrated in FIG. 9A or 9B, for example.

A diffuser 22A in conjunction with the light-emitting characteristics ofLEDs 52 causes the variation in intensity of light from LEDs 52 over therear face of light modulator 20 to be smooth.

A similar effect can be obtained without a diffuser 22A by spacing lightmodulator 20 away from LEDs 52. Where light modulator 20 is spaced awayfrom LEDs 52, light from each LED 52 can contribute to illuminatingedges of the areas of spatial light modulator 20 corresponding toneighboring LEDs 52.

In cases where it is necessary that the display be viewable through alarge range of angles, such spacing can cause a parallax problem. Wherea viewer is not viewing a display head-on, as shown in FIG. 8B, theviewer may see a pixel of spatial light modulator 20 illuminated by anLED 52 which does not correspond to the pixel. For example, in FIG. 8B,area 21A corresponds to LED 52A and area 21B corresponds to LED 52B.However, due to parallax, the viewer sees pixels in area 21A as beingilluminated by LED 52B.

FIG. 8C shows an alternative construction which avoids the parallaxproblem illustrated by FIG. 8B. In FIG. 8C, a grid 122 ofreflective-walled channels 123 is disposed between array 50 and spatiallight modulator 20. In a preferred embodiment, channels 123 arehexagonal in cross section and grid 122 comprises a honeycomb structureas shown in FIG. 8D. Channels 123 could also have other cross sectionalshapes such as square, triangular, rectangular or the like. The wallswhich define channels 123 are preferably thin. Grid 122 could comprise,for example, a section of aluminum honeycomb material.

Channels 123 may be, but are not necessarily hollow. Channels 123 may beprovided by columns of light-transmitting material having walls at whichlight is internally reflected, preferably totally internally reflected.The columns may be separated by thin air gaps or clad in one or morematerials which provide an interface at which light is internallyreflected. The columns may be integral with the material in which LEDs52 are encapsulated. FIG. 8G shows an embodiment of the invention inwhich columns 123A having internally reflecting walls are integrallyformed with LEDs 52C. Columns 123A may have various cross sectionalshapes such as hexagonal, triangular, square or the like.

Light from each LED 52 passes through a channel 123. As shown in FIG.8E, some light from an LED passes straight through channel 123 and somelight is reflected from reflective walls 124 of channel 123. Theluminance at a point on spatial light modulator 20 is contributed to byboth reflected and non-reflected light. The reflected component tends tobe more intense around the edges of channel 123 while the non-reflectedcomponent tends to be more intense toward the center of channel 123. Theresult is that the uniformity with which each LED 52 illuminates thecorresponding portion of spatial light modulator 20 is improved by thepresence of grid 122. The increase in uniformity is illustrated in FIG.8F.

Grid 122 is spaced slightly away from spatial light modulator 20 by agap 57 (see FIGS. 8C and 8E) to avoid perceptible shadows cast by thewalls which separate adjacent channels 123 of grid 122.

The geometry of channels 123 may be varied to achieve design goals. Thewidth of each channel 123 largely determines the resolution with whichthe intensity of light falling on spatial light modulator 20 can bevaried. For a given channel width and cross sectional shape, theuniformity of illumination provided by each channel 123 can be increasedby making the channel 123 longer. This, however, reduces the efficiencywith which light is passed to spatial light modulator 20.

A reasonable trade off between efficiency and uniformity of illuminationmay be achieved by providing channels 123 which have lengths L such thatnear the channel edges non-reflected and once-reflected light componentsare each approximately half of the intensity of the non-reflectedcomponent on the axis of LED 52. One way to approximately achieve thisis to choose length L such that the angle θ between the axis of LED 52and the edge of channel 123 is equal to the half angle θ_(1/2) of theLED 52. The half angle is the angle at which the illumination providedby LED 52 has an intensity equal to one half of the intensity ofillumination in a forward direction on the axis of LED 52. Thiscondition is provided by making L satisfy the condition of equation (1),where R is the half-width of channel 123.

$\begin{matrix}{L = \frac{R}{\tan\left( \theta_{1/2} \right)}} & (1)\end{matrix}$

It is generally desirable to provide one channel 123 for each LED orother light source. In some embodiments of the invention each channel123 has a plurality of LEDs. In one embodiment of the invention eachchannel 123 has three LEDs of different colors, for example, red, greenand blue. In such embodiments it is important that the channel 123 belong enough that light from each of the LEDs be uniformly distributed atspatial light modulator 20 as the human eye is sensitive to variationsin color.

As described above, with reference to FIGS. 7 and 7A, light modulator 20is preferably illuminated in a manner such that the illumination oflight modulator 20 by LED array 50 changes smoothly with position onlight modulator 20. This can be accomplished by providing LEDs 52 in LEDarray 50 which emit light in patterns which overlap somewhat on lightmodulator 20. The light emitted by each LED 52 may be characterized by aspread function such that the variation of the intensity of light froman LED 52 incident on light modulator 20 is the convolution of arectangular profile and the spread function. The spread functionpreferably has a full width at half maximum in the range of 0.3×d₂ to3×d₂, where d₂ is the center-to-center spacing on light modulator 20between the illumination patterns of adjacent LEDs 52 on light modulator20. A diffuser 22A (shown in dashed lines FIG. 8) may be interposedbetween array 50 and light modulator 20 to broaden the illuminationpatterns of LEDs 52 on light modulator 20.

For some applications it may be desirable to provide a display on whichthe level of illumination of closely spaced pixels may be greatlydifferent. This may be achieved, at the cost of some smoothness, byconfining light originating from each of the light sources of array 50so that the illumination patterns of adjacent light sources on lightmodulator 20 do not overlap significantly. This may be achieved, forexample, by providing light barriers 56 which limit the spread of lightfrom each of the light sources of array 50 as shown in FIG. 9C. Withlight barriers 56, each light source of array 50 illuminates onlycorresponding pixels of light modulator 20. This may also be achieved byproviding light sources 52 which project substantially non-overlappingillumination patterns onto light modulator 20. In either case, theresulting image displayed to a viewer may appear somewhat sharper thanin embodiments wherein light from each light source 52 is permitted tospread sufficiently that it provides significant illumination to somepixels corresponding to adjacent light sources. In many cases,limitations of the human eye will make this increased level of sharpnessunnoticeable.

Light modulator 20 may be a monochrome light modulator. In thealternative, light modulator 20 may be a high resolution color lightmodulator. Light modulator 20 may comprise, for example, a LCD array.Display 60 can be quite thin. For example, display 60 may be 10centimeters or less in thickness.

FIG. 10 shows a projection-type display 70 which is similar to display60 of FIG. 8 except that an array 50 of light sources 52 is imaged ontoa light modulator 20 by a suitable optical system 17.

A controller 39 may control the elements of array 50 to provide alow-resolution version of an image to be displayed on spatial lightmodulator 20. Controller 39 may control the elements of spatial lightmodulator 20 to supply features having a high spatial resolution and tootherwise correct the image provided by array 50 as described above.

One problem with using LEDs 52 as light sources in a high resolutionhigh quality display is that the brightness of light emitted at aspecific current level can vary significantly between individual LEDs.This variation is due to manufacturing process variations. Further, thebrightness of light that a LED 52 will produce tends to slowly decreasein an unpredictable manner as the LED ages. It is therefore desirable toprovide a mechanism for calibrating an LED array 50 to compensate fordifferences in brightness between different LEDs 52 in array 50.

One calibration mechanism 78 which is illustrated schematically in FIG.11 provides a light detector 80 which detects light emitted by each ofLEDs 52. Light detector 80 may be moved into different positions forcapturing light from different LEDs 52. In the alternative, a suitableoptical system may be provided to direct light from LEDs 52 to lightdetector 80. Controller 39 receives a signal 81 from light detector 80.Signal 81 indicates the brightness of light emitted by each LED 52 inarray 50 for a given current. If the brightness of light emitted by anLED 52 differs from a desired value then controller 39 determines acorrection to be applied to the current applied to each LED 52.Controller 39 subsequently applies the correction. Calibration mechanism78 may be used for initial calibration of a display. Calibrationmechanism 78 may optionally include a calibration controller 39A whichperforms some calibration tasks, such as determining a correction to beapplied to the current applied to each LED 52, and making the resultingcalibration information available to controller 39.

It is desirable to provide a calibration mechanism that does notinterfere with the normal operation of a display. One way to achievethis is to detect light which is emitted by an LED in a direction otherthan the forward direction. FIG. 11A shows a typical LED 52. Most lightemitted by LED 52 is directed in a forward direction as shown by arrow55A. A very small fraction of the light emitted by each LED 52 isemitted sideways as indicated by arrows 55B or rearwardly as indicatedby arrow 55C. Light emitted in a direction other than the forwarddirection may be termed “stray light”. One or more light detectors 80Amay be located to detect stray light from each LED 52.

A calibration mechanism 90 according to one embodiment of the inventionis shown in FIG. 11B. In calibration mechanism 90, small opticalwaveguides 82 carry stray light from LEDs 52 to a light detector 80.Only a small fraction of the light emitted by each LED 52 is captured bywaveguides 82. As long as the coupling between a waveguide 82 and thecorresponding LED 52 does not change, the proportion of the lightemitted by an LED 52 which is captured by waveguide 82 remains constant.One light detector 80A or a few light detectors 80A may be located atconvenient locations such as at edges of array 50.

FIG. 11C shows a calibration mechanism 90A according to anotherembodiment of the invention. In mechanism 90A, individual opticalwaveguides 82 are replaced by a planar optical waveguide 82A. Powerleads for LEDs 52 pass through holes 83 in waveguide 82A. One or morelight detectors 80A are located at edges of optical waveguide 82A. Lightemitted in the rearward direction by any of LEDs 52 is trapped withinoptical waveguide 82A and detected by light detector(s) 80A.

FIG. 11D shows another optical calibration mechanism 90B wherein aplanar optical waveguide 82B collects light emitted by LEDs 52 insideways directions and carries that light to one or more lightdetectors 80A.

FIG. 11E shows another optical calibration mechanism 90C wherein aplanar optical waveguide 82C collects a small fraction of the lightemitted by LEDs 52 in the forward direction and carries that light toone or more light detectors 80A. Waveguide 82C is constructed so thatsome light passing through it in the forward direction is trapped inwaveguide 82C and carried to light detector(s) 80A. To achieve this, onesurface of waveguide 82C, typically the surface facing LEDs 52 may beroughened slightly to scatter some light generally into the plane ofwaveguide 82C or some scattering centers may be provided in the materialof waveguide 82C. In the illustrated embodiment, waveguide 82C acts as aspacer which maintains a gap 57 between a grid 122 and spatial lightmodulator 20. Calibration mechanism 80C has the advantage that opticalwaveguide 82C does not need to be penetrated by holes 83 which caninterfere with the propagation of light to light detector(s) 80A.

In operation, an array 50 is first factory calibrated, for example, witha calibration mechanism 78 (FIG. 11). After, or during, factorycalibration LEDs 52 are turned on one at a time with current at acalibration level. Light detector(s) 80A are used to measure stray lightfor each LED 52. Information about the amount of stray light detectedfor each LED 52 may be stored as a reference value. Over the life of LEDarray 50, mechanism 90 can be used to monitor the brightness of each LED52. Depending upon the application, such brightness measurements may bemade at times when the display is initialized or periodically while thedisplay is in use. Brightness measurements of one or more LEDs 52 may bemade in intervals between the display of successive image frames.

If mechanism 90 detects that the brightness of an LED 52 has changedover time (typically as indicated by a decrease in the amount of straylight detected by light detector(s) 80A in comparison to the storedreference value) then controller 39 can automatically adjust the currentprovided to that LED 52 to compensate for its change in brightness.

A calibration mechanism 90 can also be used to detect failures of LEDs52. Although LEDs 52 tend to be highly reliable they can fail.Calibration mechanism 90 can detect failure of an LED 52 by detecting nolight from LED 52 when controller 39 is controlling LED 52 to be “ON”.Certain failure modes of an LED 52 or a row of LEDs 52 may also bedetected by LED driving electronics associated with controller 39. Ifthe driving electronics detect that no current, or a current having anunexpected value, is being delivered at a time when current should bepassing through one or more LEDs 50 then the driving electronics maygenerate an error signal detectable by controller 39.

Where controller 39 detects a failure of one or more LEDs 52, controller39 may compensate for the failure(s) by increasing brightness of one ormore neighboring LEDs 52, adjusting the elements of spatial lightmodulator 20 which correspond to the failed LED 52 to provide greaterlight transmission, or both. In fault tolerant displays according tothis embodiment of the invention, after failure of an LED 52, spill overlight from adjacent LEDs 52 illuminates the area corresponding to thefailed LED 52 sufficiently to make the image visible in the area.

Where controller 39 is configured to increase the brightness ofneighboring LEDs 52, controller 39 may determine the amount of increasebased in part upon the image content of the area of spatial lightmodulator 20 corresponding to the failed LED. If the image content callsfor the area to be bright then the brightness of neighboring LEDs may beincreased more than if the image content calls for the area to be dark.The resulting image quality will be degraded but catastrophic failurewill be avoided.

In some embodiments of the invention each LED 52 is dimmed or turned offduring those times when the corresponding elements of spatial lightmodulator are being refreshed. Some spatial light modulators refreshslowly enough that the refresh can be perceived by a viewer. This causesan undesirable effect called “motion blur”.

With proper timing, at those times when each row of spatial lightmodulator 20 is being refreshed, corresponding LEDs 52 can be off ordimmed. At other times the corresponding LEDs 52 can be overdrivensufficiently that a viewer perceives a desired brightness. The viewer'seye cannot perceive rapid flickering of LEDs 52. Instead, the viewerperceives an average brightness. It is typically desirable to multiplexthe operation of LEDs 52. Where LEDs are operated in a multiplexedmanner, correcting for motion blur can be performed by synchronizing themultiplexing of LEDs 52 with the refreshing of spatial light modulator52.

As will be apparent to those skilled in the art in the light of theforegoing disclosure, many alterations and modifications are possible inthe practice of this invention without departing from the spirit orscope thereof. For example:

-   -   diffuser 22 and collimator 18 could be combined with one        another;    -   diffuser 22 and collimator 18 could be reversed in order;    -   multiple cooperating elements could be provided to perform light        diffusion and/or collimation;    -   the function of diffuser 22 could be provided by another element        which both diffuses light and performs some other function. In        such cases, the other element may be said to comprise a diffuser        and an apparatus comprising such an element comprises a        diffuser;    -   the order in screen 23 of second light modulator 20 collimator        18 and diffuser 22 could be varied;    -   the signal 38A driving first light modulator 16 may comprise the        same data driving second light modulator 20 or may comprise        different data.    -   Instead of or in addition to providing measuring light output        for fixed calibration currents, calibration mechanisms 78 and/or        90 could adjust current to a LED 52 until the LED 52 provides a        desired brightness.        Accordingly, the scope of the invention includes, but is not        limited to, the substance defined by the following claims.

1. A display comprising: a light source comprising a two-dimensionalarray of light emitting elements each having a controllable lightoutput; a local dimming modulator comprising a plurality of controllableelements located to modulate light from the light source, each of theplurality of controllable elements having a controllable transmissivity;and a controller configured to estimate intensities at each of thecontrollable elements of a pattern of the light produced on the spatiallight modulator by the light emitting elements and to control thetransmissivities of the controllable elements based at least in part onthe estimated intensities.
 2. A display according to claim 1 wherein thelocal dimming modulator comprises a spatial light modulator.
 3. Adisplay according to claim 1 wherein the display is part of arear-projection device.
 4. A display according to claim 1 wherein thecombination of light emitted form the light source and modulation causedby the local dimming modulator cause a displayed image to have a highdynamic range of greater than 800:1.
 5. A display comprising: a lightsource comprising a two-dimensional array of light emitting elementseach having a controllable light output; a spatial light modulatorcomprising a plurality of controllable elements located to modulatelight from the light source, each of the plurality of controllableelements having a controllable transmissivity; and a controllerconfigured to estimate intensities at each of the controllable elementsof a pattern of the light produced on the spatial light modulator by thelight emitting elements and to control the transmissivities of thecontrollable elements based at least in part on the estimatedintensities.
 6. A display according to claim 5 wherein the controller isconfigured to estimate the intensities by, for each of the intensities,combining contributions to the intensity from a plurality of the lightemitting elements.